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Ada  evolved  from  a  desire  within  the  Department  of  Defense  to  have  a  standard 
language  for  the  development  of  real-time  and  large  scale  systems.  In  addition  to  pro¬ 
viding  features  needed  by  those  types  of  systems,  Ada  supports  structured  program¬ 
ming,  data  abstraction,  modularity,  and  information  hiding.  Research  with  these  tech¬ 
niques  indicates  that  their  use  should  improve  the  quality  of  the  software  development 
process  and  its  product.  While,  programmers  who  are  most  familiar  with  various  assem¬ 
bly  languages  and  FORTRAN  may  use  structured  programming,  generally  they  are  not 
familiar  with  the  other  concepts.  The  problems  with  training  programmers  in  Ada  and 
its  associated  design  and  programming  methods  and  then  redeveloping  current  systems 
in  Ada  b  unknown. 

In  order  to  understand  the  effect  of  using  Ada,  the  University  of  Maryland  and  the 
General  Electric  Company  began  a  joint  project.  The  purpose  of  the  project  is  to  moni¬ 
tor  the  use  of  Ada  in  an  industrial  software  development  project.  In  particular,  we 
identify  areas  of  success  and  difficulty  in  learning  and  using  Ada  as  both  a  design  and 
coding  language.  Our  results  indicate  where  emphasis  should  be  placed  in  Ada  training 
and  in  the  development  of  toob  and  techniques  for  use  with  Ada.  We  also  identify 
metrics  used  to  evaluate  and  predict  the  cost,  quality,  and  maintainability  of  Ada  pro¬ 
grams. 

Copies  of  the  newsletters  may  be  obtained  from  Dr.  Victor  R.  Basili,  Department  of 
Computer  Science,  University  of  Maryland,  College  Park,  MD,  20742.  Feedback  regard¬ 
ing  our  approach,  goab,  and  results  b  welcome. 


*  John  Bailey  and  Elizabeth  Kruesi  are  now  with  Software  Metrics,  Inc.  Sylvia  Sheppard  is 
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Background 


Ca$e  Studf  Organization 

This  case  study  is  driven  by  a  set  of  goab  that  were  devebped  to  answer  questions 
about  the  Ada  language  and  its  use.  Setting  out  with  a  set  of  goab,  a  framework  b 
developed  for  establbhing  why  each  data  item  b  collected  and  how  it  b  used  to  evaluate 
the  project  being  studied.  Thb  approach  minimises  the  chance  that  needed  data  will 
not  be  collected  and  permits  the  interpretatbn  of  the  data  relative  to  the  goab  set.  The 
goab  also  provide  an  organizatbn  for  the  data  coUectbn  process.  The  approach  consuls 
of  six  steps: 

1)  the  development  and  categorisatbn  of  a  set  of  goab, 

2)  the  development  of  a  set  of  questbns  of  interest  or  hypotheses  based  upon  those 
goab  that  attempt  to  quantify  the  abstractions  of  the  goab, 

3)  the  development  of  metrics  and  data  dbtributions  that  answer  the  questions, 

4)  the  development  of  forms  and  other  mechanbms  for  collecting  the  data, 

5)  the  actual  data  coUectbn  process,  and 

6)  the  validation  and  analysb  of  the  data. 

[Basili,  Weiss  82]  contains  a  detailed  dbcussion  of  thb  approach  to  data  coUectbn. 

The  primary  goal  of  thb  endeavor,  to  study  an  Ada  project  in  order  to  make 
recommendations,  was  broken  down  into  more  specific  goab  to  guide  the  study.  These 
goab  were  divided  into  four  major  categories:  changes  and  effort  goab,  Ada  and  PDL 
goab,  data  collection  goab,  and  metric  goab.  Each  goal  within  a  category  was  associ* 
ated  with  a  series  of  questions  whose  answers  might  help  meet  that  goal.  These  ques¬ 
tions  did  not  include  every  question  one  might  ask,  but  they  were  meant  to  be  represen¬ 
tative  of  the  questions  of  greatest  interest.  Each  question  had  a  Ibt  of  data  sources, 
such  as  forms,  static  analyzers,  or  human  evaluations,  to  guide  the  data  coUectbn  pro¬ 
cess.  The  complete  Ibt  of  goab,  without  the  questions,  b  given  in  the  appendix. 

The  source  of  much  of  the  effort  and  change  data  was  a  set  of  forms  that  were 
devebped  to  gather  information  about  ^!Le  software  development  process.  Most  of  these 
forms  were  adapted  from  the  NASA  Software  Engineering  Laboratory  [SEL  82).  They 
were  completed  by  the  programming  team  and  validated  by  the  monitoring  team  before 
their  entry  into  an  automated  database.  The  preliminary  analysb  of  that  data  b  given 
in  the  subsection  "Effort  and  Change.”  In  addition,  several  subjective  evaluations  were 
made  by  various  people  at  various  stages  of  the  development.  That  data  has  not  yet 
been  analyzed.  FinaUy,  a  parser,  which  checks  syntax  of  both  the  design  and  code  and 
takes  rudiment^  measurements,  has  processed  the  final  product.  The  analysb  of  that 
data  b  in  the  subsection  "Ada  Use."  The  goab  and  questions  directed  how  thb  analysb 
was  done. 

Project  Development 

The  project  studied  involved  the  redesign  and  reimplementation  of  a  portion  of  a 
satellite  ground  control  system  originaUy  written  in  FORTRAN.  Four  programmers 
were  chosen  for  theii  diverse  backgrounds:  the  lead  programmer/manager  knew  the 
appUcation,  FORTRAN,  and  assembler;  the  senior  programmer  also  knew  other 
languages  such  as  COBOL,  PL/I,  Lbp,  ALGOL,  and  SNOBOL;  the  junior  programmer 


had  just  earned  a  B.S.  in  computer  science  and  was  was  fluent  in  Pascal  and  other 
block-structured  languages;  the  librarian  had  only  brief  exposure  to  FORTRAN.  They 
were  given  a  month  of  training  in  Ada  and  the  programming  practices  they  were 
expected  to  use:  design  and  code  walkthroughs  and  structured  programming.  They 
practiced  using  the  NYU  Ada/Ed  interpreter  on  sample  programs  and  began  designing 
the  system.  The  design  was  written  using  an  Adarlike  PDL  which  specifles  Ada  pack¬ 
ages  and  subprograms  and  their  interfaces  as  well  as  abstract  statements  for  program 
functions.  Although  the  PDL  is  designed  to  be  processable,  a  processor  was  not  avail¬ 
able  at  design  time.  The  design  evolved  into  Ada  code  which  was  processed  by  the 
Ada/Ed  interpreter;  however,  the  entire  project  could  not  be  interpreted  as  a  unit  due 
to  size  constraints  with  the  interpreter.  The  design  and  coding  phases  of  the  project 
extended  from  March  1082  to  January  1083.  Some  testing  of  the  system  was  done  dur^ 
ing  the  summer  of  1083  using  the  ROLM  compiler;  however,  the  entire  system  has  not 
been  tested.  In  addition,  since  there  was  no  test  plan  developed  before  or  during  the 
project,  we  cannot  evaluate  the  testing  process. 

Data  Analysis 

Effort  and  Change 

Preliminary  analysis  has  been  done  on  the  effort,  change  and  fault  data.  Given  the 
overall  expenditure  on  the  project,  relative  to  most  projects,  a  large  amount  of  time  was 
spent  on  training,  about  20%.  Also,  the  effort  spent  on  each  of  requirements  and  design 
was  greater  than  the  effort  spent  on  coding.  Little  time  was  spent  on  testing.  However, 
it  must  be  stressed  that  the  project’s  development  cycle  was  not  completed.  As  it 
became  apparent  that  a  full-fledged  compiler  would  not  be  available  for  use  on  this  pro¬ 
ject,  the  programmers’  enthusiasm  waned.  Some  low-level  procedures  were  not  written, 
and  very  little  of  the  system  was  more  than  unit  tested.  Therefore,  the  percentage  of 
time  spent  on  various  phases  may  be  misleading. 

While  the  programmers  were  given  a  more  extensive  training  program  than  might 
be  considered  normal,  it  should  be  noted  that  Ada  training  costs  will  most  likely  be 
higher  than  average  and  must  be  considered  when  planning  early  developments  using 
Ada.  In  addition,  many  of  the  concepts  incorporated  into  Ada  were  not  used  by  the 
programmers.  For  example,  even  though  data  abstractions  and  their  use  were  taught 
during  the  training  program,  they  were  not  used  in  the  early  work  and  were  used  in 
later  coding  only  after  an  understanding  of  the  project  design  and  the  concepts  of  data 
abstraction  were  reinforced.  Training  must  be  oriented  toward  the  concepts  behind  Ada 
and  how  they  afe  supported  by  the  language  rather  than  toward  the  language  with 
reference  to  the  concepts.  A  related  study  concerning  training  in  Ada  is  described  in  the 
section  ’’Training  Study.” 

After  the  training,  the  project  programmers  began  to  design  and  then  code. 

Changes  were  documented  from  the  time  that  each  piece  of  design  was  reviewed.  The 
change  report  forms  show  that  most  changes  were  design  (32%)  and  code  (61%)  changes 
and  that  there  were  few  requirements  changes  (7%).  In  addition,  most  of  the  changes 
were  fault  corrections  (57%)  and  improvements  for  clarity,  maintainability,  and  readar 
bility  (23%).  The  need  for  change  was  determined  in  less  than  an  hour  for  almost  all  of 
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the  changes,  and  the  time  to  design  and  implement  the  change  was  one  hour  or  less  for 
almost  all  of  the  changes.  Since  the  code  was  not  thoroughly  tested,  we  do  not  know 
whether  an  even  higher  percentage  of  fault  correction  type  changes  may  be  needed. 

Analysis  of  the  fault  report  forms  indicates  that  72%  of  the  faults  entered  the  sy»> 
tern  during  the  coding  stage  and  24%  during  the  design  stage.  However,  since  the 
design  was  not  machine  checked  and  the  programmers  did  not  go  back  to  the  design  to 
determine  whether  a  fault  discovered  in  the  code  was  present  in  the  design,  preliminary 
visual  analysis  indicates  these  percentages  are  probably  closer  to  50/50.  Most  of  the 
faults  were  incorrect  code  (70%)  and  incorrect  design  (10%).  The  majority  of  the  forms 
indicated  that  the  use  of  Ada  contributed  to  the  fault  and  most  of  these  were  syntactic 
faults.  Programmers  claimed  that  the  Ada  language  reference  manual  or  class  notes 
explained  the  features  clearly  in  most  eases  and  that  they  understood  the  features  but 
did  not  apply  them  correctly.  To  correct  the  fault,  programmers  usually  remembered 
how  the  features  should  be  applied  or  obtained  information  from  another  progranuner. 
Most  faults  took  less  than  fifteen  minutes  to  isolate  and  as  little  time  to  correct.  The 
activities  used  to  detect  and  isolate  faults  were  mainly  compilation,  design  reading, 
design  walkthrough,  code  reading,  or  some  combination  of  these. 

The  above  information  seems  to  indicate  that  most  of  the  faults  discovered  were 
trivial.  Without  having  done  thorough  testing,  it  b  impossible  to  say  how  many  more 
serious  and  change-resbtant  faults  may  still  ezbt  in  the  code. 

The  faults  were  abo  classified  as  language,  problem  and  clerical.  Language  faults 
were  those  which  Involved  the  syntax  or  semantics  of  a  feature  or  those  which  involved 
the  concept  behind  a  feature.  The  problem  category  involved  those  faults  due  to  a  lack 
of  understanding  of  the  approach  or  solution  domain  but  not  related  to  the  language. 
Clerical  faults  included  those  due  to  carelessness,  e.g.  typographical  faults.  Ei|^ty>six 
percent  of  the  faults  were  language  faults,  and  furthermore,  60%  of  these  were  merely 
syntax  faults.  Thb  explains  why  so  many  of  the  faults  took  so  little  time  to  correct. 
Twenty-seven  percent  of  the  language  faults  were  semantic  faults.  Most  of  the  faults 
involving  requirements  were  problem  faults,  and  most  of  the  faults  involving  incorrect 
design  or  code  were  language-related  faults. 

Several  Ada  language  features  were  involved  in  faults.  Most  common  among  these 
were  low-level  syntax  (e.g.  semicolon,  parenthesb,  assignment)  and  loops.  There  were 
also  a  considerable  number  of  faults  involving  tasks,  separate  compilation,  generics,  pro¬ 
cedures  and  functions,  parameters,  and  declarations.  A  smaller  number  of  faults  were 
related  to  exceptions,  types,  packages  and  several  other  features.  There  were  only  a  few 
concept  faults,  and  these  involved  tasking,  exceptions  and  packages.  Parameters,  gener¬ 
ics  and  compilation  units  together  accounted  for  53%  of  the  semantic  faults.  These 
results  suggest  that  further  training  m  the  concepts  of  Ada,  along  with  a  language-based 
editor,  might  eliminate  many  of  the  type  of  faults  found  in  thb  project. 

Ada  lJ$t 

All  of  the  design  and  code  has  been  processed.  There  are  11145  lines  of  Ada  source 
(including  comments)  and  7406  lines  of  PDL  source,  some  of  which  evolved  into  Ada 
source.  The  Ada  code  eonsbts  of  2013  statements  (1064  declarations  and  1840 
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executable  statement*).  There  are  SO  program  units  (packages,  tasks,  or  subprograms), 
18  of  which  are  packages. 

Early  design  reviews  showed  that  the  design  was  functional  rather  than  object- 
oriented.  This  subjective  opinion  is  supported  by  an  analysis  of  the  packages.  Of  the 
eighteen  packages,  two  were  common  blocks  of  definitions,  three  were  libraries  of  func¬ 
tions,  eleven  were  encapsulated  data  types  with  private  types  and  operations,  and  the 
remaining  two  had  defined  types  but  made  the  representation  of  the  type  visible.  Nine 
of  the  packages  defining  encapsulated  types  were  device  drivers,  one  encapsulated 
mathematical  functions  for  different  types  of  data,  and  the  remaining  package  definition 
had  no  body.  Device  drivers  and  math  libraries  are  used  in  existing  software  systems. 

No  new  fully  encapsulated  types  were  declared.  Therefore,  the  programmers  did  not 
seem  to  find  new  abstract  data  types  [Gannon,  et  al.  83]. 

One  reason  for  use  of  a  functional  design  might  be  that  the  requirements  are 
detailed  and  functionally  oriented.  It  was  probably  easier  for  the  programmers  to  design 
the  system  functionally  based  on  those  requirements  than  to  abstract  back  from  the 
requirements  to  a  level  where  they  could  see  other  design  alternatives  (Duncan,  et  al. 

84).  In  addition,  since  the  programmers  had  more  experience  with  FORTRAN  than  any 
other  language,  they  may  have  been  constrained  by  their  previous  language  experience 
[Booch  81).  Training  for  alternative  design  approaches  and  other  software  engineering 
concepts  supported  by  Ada  must  come  early  in  development.  Thu  training  probably 
should  precede  training  in  the  Ada  language,  since  it  impacts  the  early  design  decisions 
and  perhaps  the  requirements  analysis  phase. 

Two  of  the  goals  of  the  project  (II.2  and  II.5)  relate  to  the  use  of  the  Ada  language. 
As  a  first  step,  we  have  examined  each  programmer’s  use  of  executable  statements.  Of 
the  Ada  executable  statements  (1840),  18%  (301)  were  written  by  the  lead 
programmer/manager,  43%  (795)  by  the  senior  programmer,  36%  (671)  by  the  junior 
programmer,  and  5%  (82)  by  the  librarian.  Any  comparison  of  language  use  will  prob¬ 
ably  not  include  the  librarian  because  he  wrote  relatively  few  executable  statements.  In 
discussing  each  programmer’s  use  of  Ada,  we  indicate  which  percentage  of  each 
programmer’s  executable  statements  u  involved  in  order  to  normalize  the  data. 

The  librarian  was  the  only  team  member  to  use  a  ducernibly  limited  subset  of  Ada 
executable  statements.  He  used  assignments  (40%),  ifs  (20%),  returns  (16%),  loops 
(13%),  and  raised  exceptions  (2%).  Thb  use  seems  to  be  appropriate  for  the  subpro¬ 
gram  he  wrote.  The  other  programmers  used  almost  every  type  of  executable  state¬ 
ment.  The  code  statement  was  probably  not  appropriate  for  this  application,  and  they 
avoided  the  goto  statement  as  well.  However,  the  lead  and  senior  programmers  used  10 
and  20  (3.3%  ahd  2.5%)  exit  statements  respectively.  The  exit  can  be  considered  a  res¬ 
tricted  goto.  Only  the  senior  programmer  used  the  abort  statement,  and  the  lead  pro¬ 
grammer  used  14  (4.7%)  pragmas  whUe  the  junior  programmer  used  2  (0.3%). 

Little  distinction  between  programmers  can  be  made  using  this  data  at  this  level  of 
analysis.  We  are  investigating  more  detailed  measures  of  language  and  data  use.  We 
also  will  try  to  develop  further  measures  of  the  use  of  Ada  concepts  such  as  exception 
handling,  tasking,  and  abstraction  (Basili,  Kats  83).  As  our  analyzer  becomes  more 
sophisticated,  we  hope  to  further  characterize  the  use  of  Ada  on  this  and  other  projects. 
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Training  Study 

In  a  related  empirical  study,  we  compared  two  approaches  to  teaching  the  Ada 
language.  The  goal  was  to  discover  an  effective  way  to  teach  students  the  use  of  Ada  as 
a  vehicle  for  applying  information  hiding  and  data  abstraction  to  software  development. 
The  fifty-four  participants  in  the  study  were  enrolled  in  an  advanced  undergraduate 
Ada  class  at  the  University  of  Maryland.  Baseline  data  was  gathered  on  every  student, 
including  programming  aptitude  scores.  The  class  was  then  randomly  divided  into  two 
sections.  One  section  was  taught  the  language  features  first,  approximately  in  the  order 
that  they  are  presented  in  the  language  manual.  They  were  then  shown  how  packages 
can  be  used  to  encapsulate  objects,  resources,  and  types  when  a  system  is  first  designed. 
The  other  section  was  taught  these  principles  of  encapsulation  first  and  used  the  Ada 
package  to  express  designs  before  the  lower-level  language  features  were  presented. 
Eventually,  the  same  set  of  lectures  was  presented  to  both  sections. 

We  initially  hypothesized  that  the  section  which  learned  design  first  would  produce 
more  modifiable  programs.  However,  the  lack  of  complete,  executable  examples  during 
the  entire  first  half  of  the  course  appeared  to  hamper  a  complete  understanding  of  the 
concepts.  Ultimately,  the  high  variability  among  the  students  masked  any  large 
differences  between  the  sections.  However,  some  interesting  differences  in  the  correla¬ 
tions  between  background  data  and  the  success  of  the  students  in  each  section  were 
revealed.  This  experience  suggests  that  the  optimal  approach  would  probably  involve 
tailoring  a  curriculum  to  each  student’s  background  and  experience.  However,  a  combi¬ 
nation  of  the  two  approaches,  where  complete  examples  are  presented  with  emphasb  on 
design  considerations,  might  be  appropriate  even  when  teaching  professional  program¬ 
mers. 

Conclusions 

The  Ada  language  is  a  medium  for  supporting  certain  design  concepts.  It  is  impor¬ 
tant  that  those  concepts  be  taught  and  motivated,  possibly  even  before  the  language  is 
taught.  Training  should  be  tailored  to  the  past  experience  of  the  programmers.  The 
programmers  on  this  project  had  trouble  with  data  abstraction  and  information  hiding 
and  distinguishing  between  detail  and  precision  particularly  when  designing.  Only  after 
the  project  was  complete  did  they  understand  the  importance  of  methodology  and  how 
it  should  be  used.  Their  overall  design  was  more  like  than  unlike  a  FORTRAN  system 
design.  However,  the  requirements  were  already  functional. 

In  this  project,  the  programmers  used  most  of  the  language  features  but  not  neces¬ 
sarily  as  they  were  intended  by  the  language  designers.  There  were  a  large  number  of 
language  errors  made,  and  these  errors  were  syntactic,  semantic,  and  conceptual.  Most 
of  the  errors  involved  the  more  Ada-specific  features.  Due  to  the  learning  curve,  we 
were  unable  to  judge  the  impact  of  Ada  on  costs,  schedules,  or  milestones.  However,  it 
is  clear  that  many  support  toob  are  needed.  These  toob  include  a  structured  editor, 
data  dictionaries,  call  structure  and  compilation  dependency  toob,  cross  references,  and 
other  means  of  obtaining  multiple  views  of  the  system.  In  addition,  a  PDL  processor 
with  interface  checks,  definition  and  use  relation  Ibts,  and  metrics  would  be  helpful  in 
the  early  stages  of  development. 


Further  Research 

Some  further  analysis  will  be  done  with  this  data.  The  design  and  code  will  be  stu> 
died  to  determine  whether  previous  experience  influences  a  programmer’s  use  of  Ada. 
Since  the  project  was  not  flnished  and  the  product  not  tested  fully,  we  expect  few  con* 
Crete  results  from  this  data.  However,  we  plan  to  use  thb  data  to  aid  in  the  evaluation 
of  analysis  tools  we  will  develop.  After  building  these  toob,  we  plan  to  look  at  other 
projects  developed  in  Ada  for  further  indications  of  the  effect  of  Ada.  Recommendsr 
tions  wUl  then  be  made  on  which  metrics  are  most  useful,  what  aspects  of  training  must 
be  stressed,  and  what  influence  the  use  of  Ada  might  have  on  the  software  development 
process.  We  encourage  comments  on  all  aspects  of  thb  project  and  will  continue  to 
publbh  newsletters  or  papers  concerning  our  results. 
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Appendix 

The  purpose  of  these  goals  b  to  direct  the  study  of  thb  Ada  project.  Complete 
copies  of  the  Ibt  of  goab  and  questions  may  be  obtained  from  the  authors. 

I.  Changes  and  Resources 
I.l:  Characterize  the  effort  in  the  project. 

1.2:  Characterize  the  changes. 

1.3:  Characterize  the  faults. 

1.4:  Characterize  Ada  faults. 

1.5:  Characterize  the  other  faults. 

l. 6:  Characterize  the  non>error  changes. 

n.  Ada  and  PDL/Ada 

n.l:  Evaluate  the  effect  of  using  an  Ada-like  PDL  with  respect  to  the  goab  of  a  PDL. 
n.2:  Determine  which  subsets  of  Ada  features  are  used  naturally. 

II.3:  Determine  the  effect  of  using  an  Ada-like  PDL  when  Ada  b  the  language  of 
implementation. 

II.4:  Determine  how  Ada  works  for  thb  application. 

11.5:  Characterize  the  programmers  and  associate  their  backgrounds  with  their  use  of 
Ada. 

n.6:  Determine  whether  there  are  aspects  of  Ada  that  contribute  positively  to  the 
design  and  programming  environment. 

HI.  Data  Collection 

m. l:  Evaluate  the  data  collection  and  validation  process. 


IV.  Metrics 

IV.  1:  Select  a  set  of  static  metrics  for  the  APSE. 

IV.2:  Develop  a  set  of  size  metrics  for  the  APSE. 

IV.3:  Develop  a  set  of  control  metrics  for  the  APSE. 

IV.4:  Develop  a  set  of  data  metrics  for  the  APSE. 

IV.5:  Select  a-eet  of  dynamic  metrics  for  the  APSE. 

rV.6:  Develop,  a  set  of  test  coverage  metrics  for  the  APSE. 

IV.7:  Develop  a  set  of  execution  metrics  for  the  APSE. 

IV.8:  Select  a  set  of  software  development  process  metrics  for  the 

IV.O:  Determine  the  effectiveness  of  the  predictive  power  of  certain  measures  during 
development. 

IV.  10:  Develop  a  subjective  evaluation  system  for  evaluation  of  program  and  design 
characteristics  that  are  not  practically  or  easily  measured  in  other  ways. 

IV.ll:  Provide  a  data  base  for  future  Ada  projects  to  be  used  to  predict  some  proper¬ 
ties  of  those  projects. 


-a- 


